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As students enter postsecondary education, placing them into courses that meet their 
skill level, provide challenge, and promote forward progress toward a credential is key. 
Traditionally, community colleges and other broad-access institutions have assessed 
college readiness in math or English using a single standardized placement test score. 
However, more and more institutions are moving toward multiple measures assessment 
systems, which combine two or more indicators of student preparation — like placement 
test scores, high school records such as GPAs, and noncognitive assessment scores 
— to determine whether a student will be required to take a developmental course. 
Evidence suggests that the use of multiple measures assessment can be one of the most 
promising avenues for improving student success." Indeed, a range of multiple measures 
assessment models has been shown to increase the rate of placement, enrollment, 
and completion in college-level courses as compared to single test-based systems.’ 
Importantly, regardless of the specific model or measures used, the most effective and 
equitable multiple measures assessment systems are those that move more students — 
particularly students of color and low-income students, who are disproportionally referred 
to developmental education programs — into college-level courses and supports.° 
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State systems have played an increasingly prominent role in encouraging and supporting colleges 
to implement effective developmental education reforms, and more have begun to recommend 
or require multiple measures assessment for placement.’ Implementing multiple measures 
assessment presents challenges, including garnering buy-in for granting more students access 
to college-level courses, collecting and using institutional data to make decisions on the most 
effective and equitable process for determining placement, and marshaling the resources to 
redesign placement practices and train staff accordingly.° Enacting these changes in a resource- 
constrained environment in which institutions are tasked with an ever-growing number of 
student success initiatives presents an additional challenge. State systems can mitigate these 
challenges and promote effective practices by adopting strategic policies and providing supports 
to colleges. 


Perhaps counterintuitively, the onset of COVID-19 created opportunities for state systems 
to facilitate institutional adoption of multiple measures assessment. In spring 2020, as 
large numbers of colleges moved to remote learning and work, it was often infeasible to 
continue offering in-person, proctored placement tests. While institutions may have continued 
administering the assessments remotely, either through online remote proctoring services (e.g., 
Examity) or through local remote online proctoring via video conferencing services (e.g., Zoom), 
in many cases testing capacity was significantly reduced. As a result, institutions sought out new 
ways to assess and place students. Through changes to policy and re-allocation of resources, 


TYPES OF MULTIPLE MEASURES ASSESSMENT MODELS 


Decision Rule systems place students according to a series of “if-then” statements dependent upon their scores 
on multiple measures. Typically, if a student scores above a specified threshold on at least one measure, then 
they receive a college-level placement. If a student scores below the specified threshold on every measure, then 
they are placed into a developmental prerequisite or corequisite course. 


Decision Band systems place students according to where they fall relative to a specified range of scores on 
a certain measure. Students with scores above or below the specified range are placed as they would have 
been under a traditional placement system. Students with scores above the range receive a college-level 
placement, and students with scores below the range receive a developmental prerequisite or corequisite 
placement. Students with scores that fall within the specified range are further evaluated according to 
additional criteria. 


Algorithm systems use historical data to determine how well different factors (placement test scores, high 
school GPAs, time since high school graduation, etc.) predict student success in college-level courses. 
Using the results of those analyses, a placement algorithm is developed that combines and weights 
incoming students’ values on selected measures to establish a placement score for each student. The 
placement score is typically interpreted as the probability that a student will pass a college-level course in a 
given subject. 
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four states — Indiana, Virginia, Texas, and Washington — supported large-scale changes to 
placement practices. These states exemplify how systems and institutions can rapidly adapt to 
overcome common challenges associated with implementing multiple measures assessment. 
In the short case studies that follow, based on interviews with system leaders as well as 
college administrators, faculty, and staff, we summarize each state’s pre-COVID assessment 
and placement policies and practices and describe the system-level actions taken to mitigate 
common multiple measures assessment implementation challenges: (1) establishing buy-in, 
(2) providing implementation support and resources, (3) combatting initiative fatigue, and (4) 
establishing data-driven, continuous improvement evaluation processes. 


EVALUATION OF MULTIPLE MEASURES ASSESSMENT IN NEW YORK STATE 
SHOWS POSITIVE EFFECTS ON STUDENT OUTCOMES 


Working with the State University of New York (SUNY), CAPR researchers conducted a randomized 
controlled trial to measure the effects of multiple measures assessment at seven SUNY community 
colleges. Nearly 13,000 students were assigned to a program group that was placed using an algorithm 
that considered students’ high school grades and standardized test scores, or to a control group that 
was placed using standardized test scores alone (the status quo). Among the key findings: 


+ Many program group students were placed differently than they would have been under the status quo 
system; most were placed higher. In math, 16 percent of program group students were “bumped up” to 
a college-level course; 10 percent were “bumped down” to a remedial course. In English, 44 percent 
were bumped up and 7 percent were bumped down. 


* In English, program group students had higher rates of completion of a college-level course. While gains 
declined over time, program group students were still 5.3 percentage points more likely to enroll in 
and 2.9 percentage points more likely to complete a college-level English course than their peers 
through the third term. 


+ In math, program group students initially had better rates of course completion, but these gains were not 
sustained over time. In comparison to their peers, program group students had modestly higher rates 
of enrollment in and completion of a college-level math course in the first term, but these rates faded 
and then disappeared. 


* When broken down by gender, Pell recipient status, and race/ethnicity, all student groups benefited from 
placement using the algorithm (except for men in math). There were higher course completion rates in 
English for Black students (by 7.1 percentage points), Pell recipients (by 4.5 percentage points), and 
women (by 4.6 percentage points) in the program group. 


+ Students were strongly affected by their alternative placements. The subset of the program group that 
was bumped up into college-level courses was 8-10 percentage points more likely to complete a 
college-level math or English course within three terms. Program group students who were bumped 
down into developmental courses were 8-10 percentage points less likely to do so. 
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Case Studies 


Indiana 


At Ivy Tech Community College, the singly accredited community college in the state of 
Indiana, the pandemic prompted full-scale implementation of EdReady. 


Since 2014, in conjunction with lvy Tech Community College's scaling of its redesigned 
developmental math and English courses and its introduction of math pathways, the statewide 
community college in Indiana has been implementing an assessment system wherein students 
select the specific measures and processes used to determine their placement level. Options 
for placement have included using high school GPA, SAT, or ACCUPLACER. Students may 
also elect to enroll in developmental prerequisite or corequisite courses. In 2018 at a small 
number of campuses, Ivy Tech also began offering students the option of using EdReady — 
which features an adaptive low-stakes knowledge inventory and assessment available in both 
math and English — from the NROC Project. Upon completion of the knowledge assessment, 
students may opt to use their initial results to determine their course placement or they can 
receive an individual, personalized study path composed of learning activities they can work 
on in the EdReady platform to place into a higher-level course. 


In spring 2020, in response to the COVID pandemic, 


Ivy Tech moved rapidly to scale implementation of the ° ° 
EdReady knowledge assessment across its 45 campus Colleges seeking to realize the 


and site locations. Students at all campuses are benefits of multiple measures 
offered the option to take the knowledge assessment assessment must commit to a 


ina non-proctored, self-paced environment (students placement scheme that places 
who wish to self-place or provide other measures are 


still able to do so). EdReady provides students with more students into college- 
multiple attempts to modify their placement through level courses than would have 
the assessment, which does not serve as a one-shot been placed using a single 
exam. Importantly, by providing students agency, lvy 

Tech established a more holistic process that may measure. 
allow non-cognitive factors, such as motivation, to 
play a part in the placement process. As part of the scale-up, lvy Tech strove to foster an 
asset-based mindset regarding placement with a focus on the knowledge and skills students 
already possess when they enroll in the institution. 


To rapidly scale up these new placement practices, each campus formed a cross-functional 
team composed of a small group of staff representing all the areas of the campus touched 
by the EdReady knowledge assessment, including staff involved with dual credit and 
student support services. This team worked closely with the IT department to automate the 
assessment process and to make sure that the EdReady platform, the student information 
system (Banner), and the student portal were aligned for the system's three different math 
pathways and different programs of study. System-led professional development training 
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with campus teams was designed to shift the institutional mentality away from a deficit 
perspective of assessment that focuses on revealing what students do not yet know to one 
that emphasizes what students do know. 


Colleges seeking to realize the benefits of multiple measures assessment must commit to 
a placement scheme that places more students into college-level courses than would have 
been placed using a single measure. This may require a shift in mindset among college 
stakeholders about how many students are prepared for college-level work. Efforts to include 
more faculty and staff responsible for enacting new policies and practices in planning 
activities provides an efficient way to help college personnel understand new procedures 
as well as new ways of thinking about the purpose of assessment. Ivy Tech's engagement 
of campus-level cross-functional planning teams and its emphasis on recognizing the 
knowledge and skills that incoming students have already developed highlight one strategy 
for mitigating the challenges in implementing multiple measures assessment. 


Virginia 
During the pandemic, all institutions in the Virginia Community College System (VCCS) 


began using self-reported high school GPA as the primary placement measure for recent 
high school graduates. 


Since 2013, VCCS policy has allowed the state’s community colleges to use multiple 
measures assessment; however, this policy required the use of at least one standardized 
test score (from, e.g., the ACT, SAT, or Virginia Placement Test) to place students into first- 
year math and English courses. During the 2019-20 academic year, VCCS began a pilot at 
eight community colleges to use self-reported cumulative high school GPA and the highest 
level math course taken to place recent high school graduates.’ Benchmarks were identified 
for three placement tiers: gateway math and English courses (for those with a GPA of 3.0 
or above), gateway courses with corequisite course supports® (for those with a GPA of 2.0- 
2.99), and noncredit transitional courses (for those with a GPA below 2.0). Students can use 
standardized test scores to override their GPA placement and bump themselves up into a 
higher placement category. As part of the pilot, colleges offered students math pathways 
in quantitative reasoning, statistics, and pre-calculus. 


The pilot colleges also implemented a guided self- vccs provided significant 


lacement process for students who graduated more ° 
/ J ee guidance and resources to 
than five years ago. Default placement for older 


students is in gateway courses, but advisors can colleges, including student 
recommend corequisite supports or other courses if questionnaires to help 


a student feels it would be helpful. nontraditional students 


In spring 2020, when in-person testing was no longer place themselves. 


an option, the system waived the 2013 placement 
policy mandating the use of at least one standardized test for all remaining non-pilot colleges. 
Those colleges (only some of which had corequisite courses) asked VCCS for guidance on 
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how to rapidly implement and scale the new high school GPA approach. In response, the 
system provided significant guidance and resources, including student questionnaires to 
help nontraditional students place themselves, a memo to colleges on conducting direct 
enrollment with high school GPA and test scores, and a set of resources hosted in the system's 
learning management system for college administrators to facilitate resource sharing 
between pilot schools and the rest of the colleges. Though placement looked different at 
colleges that did not yet offer corequisite courses, the system helped each college adapt 
the direct enrollment model to the courses they were offering. Now, the system plans to 
formalize the process of resource sharing and mentoring as it implements the other elements 
of the direct enrollment pilot at the rest of the VCCS institutions: corequisite courses and 
math pathways. For students entering in fall 2021, seven colleges in a second cohort will 
implement direct enrollment with corequisite courses using high school GPA and test scores 
and partner with a mentor college of similar size and student composition from the pilot. The 
second cohort will then assist a final cohort of eight VCCS colleges in implementing the new 
direct enrollment and corequisite policy for students entering in fall 2022. 


The adoption of multiple measures assessment is a labor-intensive and time-consuming process 
that often requires colleges to mobilize or reallocate valuable resources to implementation 
efforts. It is not uncommon for colleges to seek outside assistance in the development of 
multiple measures assessment, either in the form of technical assistance from consultants 
and researchers or by collaborating with colleges who are further along in the implementation 
process. Virginia was able to quickly pivot from pilot to a full-scale adoption of the use of high 
school GPA by taking advantage of resources developed during the pilot and sharing those out 
across the system. The centralization of the system helped smooth out the quick change. 


Texas 


During the pandemic, the Texas Higher Education Coordinating Board (THECB) granted a waiver 
for students without access to placement testing so that institutions may use other indicators 
such as high school GPA to determine best placement options for entering students. 


In fall 2003, the Texas legislature passed the Texas Success Initiative (TSI), which requires 
every non-exempt student entering a Texas public college or university to demonstrate 
readiness for enrollment in entry-level college courses using a state-approved assessment. 
While the legislation allows for a number of ways that students can be exempt from this 
requirement (e.g., by meeting qualifying standards on the SAT, ACT, or the high-school- 
administered State of Texas Assessments of Academic Readiness), in 2018 almost 40 
percent of Texas students who were assessed for college readiness received a placement 
based on their Texas Success Initiative Assessment (TSIA) scores. 


In March 2020, Texas colleges and universities discontinued in-person instruction due to 
the COVID pandemic. In response, the Commissioner of Higher Education approved a TSI 
waiver for all non-exempt students who did not have access to the TSIA through the 2021- 
22 academic year. In place of the TSIA, the THECB recommends that institutions use other 
indicators, such as high school GPA, high school course-taking records, and evidence on 
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noncognitive factors, to help them determine placements for their students. In many cases, 
colleges and universities have opted to implement a decision-rule system that integrates 
measures such as overall high school GPA and highest course taken in a subject. 


The THECB has not yet determined what assessment The data being collected will 
and placement policies will be put in place at the permit the THECB to assess 


conclusion of the TSIA waiver period. But the THECB h lati hip b h 
is using this period of forced experimentation to the relationship between the 


better understand the impact that alternative systems use of particular multiple 


can have on students and how that understanding can measures assessment 
be used to better inform future assessment policies. 
systems and student 


In many colleges, placement data have not been outcomes. 

collected or linked to student outcomes, which stymies 

efforts to evaluate the performance of specific practices or policies. In contrast, the THECB will 
require all public institutions in Texas to report the criteria used during the TS! waiver period. 
In addition, the THECB will work closely with 13 institutions to track the specific placement 
criteria used to place each student. These data will be collected using the THECB’s traditional, 
though modified, reporting protocols, as well as through new surveys intended to capture a 
greater level of detail around the measures and thresholds used in placement decisions. The 
data being collected will permit the THECB to examine the utility of a variety of measures, 
better understand the impact that specific cut scores can have on the composition of student 
placement into college-level courses, and assess the relationship between the use of particular 
multiple measures assessment systems and student outcomes. Importantly, the THECB plans 
to use these data to better inform placement processes after the TSI waiver period expires. 


Washington 

In response to the pandemic, in Washington, a state with a loosely federated higher education 
system, the Student Success Center provided webinars and coaching to colleges to support the 
development and refinement of multiple measures assessment implementation plans. 


For nearly a decade, community and technical colleges in Washington State have used 
multiple measures to place students into developmental or college-level courses in math 
and English. While institutions have significant local control, statewide agreements have 
established placement reciprocity across institutions and provided guidance for using 
the Smarter Balanced Assessment, a high school Common Core exam, for placement. In 
addition, the system has encouraged its colleges to use additional measures like high school 
transcripts. Prior to the pandemic, at least 27 of the 34 community and technical colleges 
used high school transcripts to some extent for placement; however, these policies often 
were specific to local high schools and required grades in specific courses to be received 
within a short timeframe. Thus, they often impacted small numbers of students. 


In spring 2020, as the colleges were faced with devising alternative placement approaches, 
the state’s Student Success Center deployed resources to support institutions in adopting 
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thoughtfully designed, sustainable multiple measures assessment practices. The Success 
Center also hosted a four-part webinar series during the summer of 2020 to help colleges 
reflect upon, build on, and refine their quickly enacted spring placement strategy for fall 2020. 
The webinars covered the use of high school transcripts, guided self-placement practices, 
and advising practices, among other topics. Rather than endorsing a particular multiple 
measures assessment approach, the Success Center emphasized that colleges should make 
implementation decisions with the goal of ensuring greater and more equitable access to 
college-level courses. 


As a follow-up to these webinars, the Success Center is using funds from College Spark, a 
Washington-based foundation, to engage a cohort of 12 colleges that have been working 
together for over two years to plan and implement guided pathways reforms.’ In the context of 
COVID, teams from these institutions are being asked to explore their placement philosophies, 

how onboarding practices can be designed with equity in mind, and how multiple measures 
assessment strategies can be implemented in ways that align with ongoing curricular and 
programmatic reforms. In fall 2020, the teams were convened for an interactive virtual 
session on multiple measures assessment and . : 

received coaching from Success Center staff; they will The Integration of reform 


reconvene for a second session in January 2021 to strategies can help stave 
share their progress. Lessons learned from these early off initiative fatigue, which 
adopters will be shared with the rest of the institutions F ; js 
sth ciistatiee is particularly important 

in a resource-constrained 


Washington is one of many states that have begun " 
environment. 


to implement ambitious, large-scale reforms, such 
as guided pathways, in an effort to improve student 
success and reduce educational disparities. In this fast-moving reform environment, initiative 
fatigue presents a persistent risk to effective implementation of discrete interventions. 
Washington chose to make reforms to placement practices a key priority for institutions 
given COVID's disruptions to standardized testing. To integrate their efforts with current 
reforms already underway, they utilized their guided pathways cohort infrastructure to 
support planning for multiple measures assessment implementation. This integration of 
reform strategies can help stave off initiative fatigue, which is particularly important in a 
resource-constrained environment. 
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Conclusion 


The COVID-19 pandemic has presented postsecondary systems and institutions with a myriad 
of challenges, including an abrupt transition to virtual instruction and remote service delivery, 
declining student enrollments, increasing demands on support services as students navigate 
schooling in the midst of so many unknowns, and significant budgetary constraints. These 
challenges have exacerbated gaps in opportunities, with low-income students and students 
of color bearing the disproportionate burden of the virus and the associated barriers to 
educational opportunity. At the same time, the pandemic has also created opportunities for 
institutions to decrease their reliance on standardized assessments. This can serve to help 
more students enroll in college-level courses sooner, with the aim of reducing disparities in 
Outcomes and improving student success. 


However, given the various choices institutions must make about measures and cut points, 

multiple measures assessment must be enacted with thoughtful attention to implementation. 

Without careful planning, institutions implementing multiple measures assessment may not 

increase access to college-level courses. These case 

studies illustrate that state systems have a pivotal State systems have a pivotal 


role to play in bolstering college course taking by role to play in bolstering 


i licy to reduce barriers to college-level a 
using policy to re uce arriers to college eve course college course taking by 
enrollment and providing resources and guidance on 


placement practices. Through formative evaluation, reducing barriers to college- 
reform coordination, professional development, level course enrollment 


and resource sharing, state systems can support and by providing resources 
institutions to implement multiple measures 


assessment approaches that narrow opportunity gaps and guidance on placement 
and increase student success. As institutions across practices. 

the country implement, refine, and scale alternative 

approaches to determining student readiness for college-level courses, leaders, reformers, 

and researchers are presented with new opportunities to advance our understandings of how 
placement policies and practices can improve student outcomes. 
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TABLE 1. Overview of Modifications to Student Assessment Practices Due to 
COVID Pandemic 


Number of Public 
Two-Year Institutions 


Governance Model Change in Practice 


Centralized 


(Ivy Tech Community College dbeleeltocorlelezeh ce 


Indiana 45 aunfeenenltwGheat an assessment option at all 
© Seis y campuses 
institutions.) 
ein Gonmani Scaled use of self-reported 

Virginia 23 y GPA as primary placement 
College System oversees all 
two-year institutions.) Measure alliealledes 
Decentralized 
(The Texas Higher Education Waived placement test for 
Coordinating Board is the students with no access to 

Texas 63 state agency for all public testing, allowing institutions 
institutions, including both to use other indicators to 
two-year and four-year determine placements 
institutions.) 
mecenialized Provided coaching and 
(The State Board for implementation supports 

Washington 34 Community and Technical leallages io WsElyy ey ‘ 
Colleges coordinates and refine campus-based multiple 
directs two-year institutions.) Misemeagaas peste 

policies 
Notes 
Barnett & Kopko (2021). 


Barnett, Kopko, Cullinan, & Belfield (2020); Cullinan, Barnett, Kopko, Lopez, & Morton (2019). 
Bahr (2010); Chen et al. (2020); Chen & Simone (2016). 

Rutschow, Cormier, Dukes, & Cruz Zamora (2019). 

Barnett et al. (2018); Cullinan et al. (2019). 


In the summer of 2020, the Education Commission of the States funded exploratory work to help 
CAPR plan for the next generation of research on developmental education. To understand current 
reform trends, the impact of COVID-19, and policymaker and practitioner perspectives on priority 
research questions, we interviewed 107 system leaders and institutional administrators, deans, 
faculty, and staff from 14 states and Washington, DC. The case studies are based on selections from 
this research. 


7. Babb (2019). 


8. Corequisites allow students with a developmental referral to enroll directly in college-level math or 
English while co-enrolling in a learning support course. 


SSE OP Ss 


9. Guided pathways is a national reform movement in which colleges fundamentally redesign their poli- 
cies, programs, and services to better support student success (Jenkins, Lahr, Fink, & Ganga, 2018). 
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